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T. INTRODUCTION ANI SUMMARY 


A. BACKGROUNT 


The Kalman filter 15 8 recursive HKayesian least-sauares 
estimator of en m-dimensional system state vector hased on an 
m-dimensional measurement vector. The filter may oreraete in 
3 J-dimensionsal coordinate system where Jims Jin. The bhasic 
sssumretion is thet each dimension of the coordinate system 
varies sccording to 3 kth order Gsauss-Markov Frocess. The 
Kalman Filter was develored in the early 1950’%s by Kalman and 
Rucy Crefs. 1 and 2], 

/ 

The Kalman filter may be used to obtain an ortimael 
estimate of the rresent state, @ rrediction of future states, 
and/or smoothed estimates of rast states. The current stste 
estimate is senerally used to determine an ortimsl control 
ineut. Future state estimates sre used to determine ortimum 
eresent rolicy. Smoothed rast state estimates are wused for 
data analysis and model buildings. Thus the rotential areas of 
arrelication sran the field of time series analysis. 

Are lications of the Kalman filter ere nmumerous and the 
theory 18s hpeinsg continually develored = and “xtended. Ari 
overview of the develorment of linear filtering theors and an 

“tensive bhihbliodrarhy may be found in Kailath Cref.3]. A 
reasonably clear -Fresentation of theory and arrlications 1s 


contained in Gelb Cref.4]. 





Ferhers the widest and most successful agrerlication of 
Kalman filtering has heen to venicle tracking and control. 
Clark Cref.5] has written @ rarticularly lucid descrirtion of 
the design of a filter for an anti-aircraft gun fire control 
sustem which 1s moteworthy for its clarity of Fresentation of 
the underlying theory. It is evident Crefs. 4 and SJ] that 
the desigm Process 165 heuristics and reauires extensive 
testins and analysis of candidate filter confisurationss even 
whem the rrocess 1658 well-understood and 1s bhaesed on @ mature 
technology. 

The Kalman filter nas also heen arrplieds with varying 
degrees of success» in economic models»y inventory models»y and 
even weather models. Considerable difficulty 15 encountered 
in model bHhuilding»s hecause the filter design reauires good 
estimates of the variance and covariance of moise sourcesys as 
well 8S an accurate state transition model. A Frior estimate 
of sustem state and covariance is also reauiredy which 1s 
somewhat less criticel hecause errors in the Frrior estimate 
decresse with time. These rarsemeters are often difficult toa 
determine in Hhishls random rrocesses of Questionable 
stationarity. 

The Kalman filter is derived and desisned almost entirely 
Within the time domains although Clark Cref.3S] does refer to 
the concert of filter nsndwidth. The Kalman filter is 
essentialls 3 low-rass filter with @ vers wide transition 
bands and hisdher-order filters have some amplification at the 
mid or lLlow-mid freauency ranse. In s@eneraly the stor nand 


goes mot completely attenuate high freauencies. This allows 





the filter to ettenusate hish-freauencys moise somewhat while 


still retsinins some resronmse to sudden chanses of stete, 


fee PURPOSE 


The Furrose of this thesis is to gcauaint the resder with 


the Kalman filter» to show how the choice of véerious filter 


rarsmeters affect its rerformaernces and to frrovidce design 


Mmsisnt through esnaelyesis im the freauerces domsin- The 
SBererrosch is tutorisl, and the reader is referred to some of 


the iunteresting examreles which mey be found im the 


literature, 


fee METHOR 


The freauencys resronse of several simple filter designs 
were investiseted using the Fast Fourier Transform Frosram im 
the AFL Library 2, The commuter results were Justified 
analytically for the simrlest desidms 3a scalar sinsgle-state 


filter. ierivetions sre rresented in aerendix A. 


eee LEVEL OF FRESENTATION 


Full understanding of the theory reauires 8 knowledge of 
Stochastic Frocesses thst evolve over timey as well as an 
understanding of disgitel sismal theory ain the freauency 
Jomairis The Fourier transform is @ basic tool. A full 


exraosition of the underlying theory is clearly beyond the 





score of tnis Fresentation. The reader 18 directed to Larson 
ang GSNubhert Cref.éd] for the theory of stochastic rrocesses 
ana to Hammins Cref.7?] for the theory of disitsal filterina. 
AGS Freviously mentioned», Gelo Cref.4] and Clark Cref.S5] are 
good references for the Kalman filter. KRloomfield Cref.8] 
and Brillinser Cref.9] are also @rrlicable references. Krownr 
Cref.10] and Hox anid Jenkins Cref.11] contain related 
material, 

There sre few readers who are entirely comversant with 
hoth the freauency domain and time domain 3gerroach to time 
series analysis. Nevertheless» 3 duality exists hetween the 
twory end 3 summary of the theory is rresernted,. 

Lllustrstive exameles will often be bhaesed on tracking 
models, because this is eresently the widest ares of 
eerlication of Kalman filterss and because most readers will 
find the concerts of Fositions velocity» and acceleration 
easy to understand. The concerts sre easily externdsahle to 
otner greas. For exanmrley the economist may wish to rerlace 


evelocity” with “trend". 


E. SUMMARY OF RESULTS 


The steady-state dain» bandwidths and sensitivity of the 
linear discrete Kalman filter sre shown to he comrletely 
determined oy the choice of the rrocess and measurement moise 
COVSerlances. Pal Ler rerformance or stationary or 
mearly-stationary date can he Frredicted by comrsaring the 


frequency resronse of the Frrorosed filter with 3s srectral 


ee 





analysis of the date. The wide transition dand of the 
smelitude resronse of the scalar Kalman filter can be 
sharrened by Msltirle Passes of the date throush 2 
Hhisgher-sain filter. This cam he ~asccomerlished simrely and 
recursively. The sureriority of symmetric smoothing filters 
over non-summetric filters was demonstrated. When wsed as 3 
smoother (hy usins both forward and bhackward -asses) the 
Kalman filter was as effective as 82 mon-recursive Gaussian 
filter, Higher-order filters were shown to have hisher 
handwidth and amrelification 3s the order of the filter was 
increased. A freauency domain arrroach to filter design may 
Provide additional insight and enable the desigmer to achieve 
better filter Ferformancey rarticularly when the system state 
transition model and moise covariance models are not well 


understood, 


2 





oe Oley, 


A. STOCHASTIC FROCESSES AND STATIONARITY 


A continuous stochastic Frocess X(t) 1s 8 Gaussian 
erocess if the rrobability densities of all orders sre 
multivariate Gaussian densities. Et 13 32 kth order 
Gauss~-Markov Frocess if the state st time t dereands only on F 
earlier stsetes,. If we should expand the state srace to Fk 
states, which include all derivatives ur to the (k-Ll)thy the 
future system state vector will derend only om the rresent 
state. For “amele,y if the scceleration of 8 vehicle 15 38 
first-order Gsuss-Maerkov Frocessy»y tnen the Fosition of the 
vehicle ts 23 third-order Gauss-Markov Frocess,. Howevery if 
our state srace includes acceleration and velocity as well 2s 
FOSitiony the future state of the system 15 iInderendent of 
S6ll but the frresent state. If the random acceleration Nas 
zero means and veriance one over one time increments the 
acceleration 15 a standard Wiener Frocess Wits The 
derivative of the Wiener Frocess» written dW(td» has zero 
means wunit variances and is called white Gaussian moisey» 
Which may be thought of es a4 “*zero-th order" Gauss-Markov 
Process Cref,.4]. 

The standard Wiener fr-rocess 15 mot stationsrys: hecsause 
the veriance grows linearly with time. That isy the estimate 


of a future stete based on the -rresent state hes variance 


4 


1) 





tnet is 3 linear function of time. Howevers the standard 
Wiener Frocess has stationarsys imnderendent increments. That 
is» the variance sat time (ttl) given the state at time (bt) is 
constant and inderendent of t. 

A stochastic rrocess X(t) 16 wide~sense stationary if and 
only if ait hes 38 constant mean functions and a2 correlation 


function such thet Cref.4] 


R(t, + srt,t 5) = R (t, 7t,) = R Ct -t,) 


that iss the correlation function of the Process 19 
inderendent of am arbitrary time shift s. A Gaussian Frocess 
15 strictly stationary if and only auf it 15 Wide-sense 
Stationary Cref.é]. 

The Gsuss-Markav assume tion makes rossible the 
daevelorment of theory and arrlicationss because, in seneraly 
any linear orerstion rerformed on 2 Gaussian Frocess results 
Lm another Gaussian Frocesss and the Markov rfrorerty allows 
consideration of only the rresent statery disregarding all 


FPrev1iagus states, 


BR. THE FHILOSOFHICAL CONCEFT OF STATIONARITY 


A freavency-domain analysis of 3 stochastic Frocess is 
only meaningful if the Frocess 15 stationary. If the rrocess 
were chansims over timey the srectrum would chanse over time. 
Since the srectrum cam only be analyzed bys means of data 


taken over times such analysis of 8 non-stationary Frocess 


14 





would he meaningless. Howevery Lk the Frocess 1S 
*"Quasi-stationary" » that 1S» rate axhioits stationary 
statistics for 2 Whiley then undergoes 8 changer then settles 
Gown to stationarity esainy the freaquency a@rerroach is still 
usefully although iainaeccurate over the transition reriod. As 
ar examrle»s consider an 06 a@irrelane SuhbJect to random 
accelerations due to air turbulence. An arrrorriste model 
might be 2a third-order Gauss-Markov Frrocess 2@65 lonms a5 the 
agirrelane maintains a straight Fath or turns at ~a constant 
acceleration. However, the prilot’s ineuts to initiate or 
terminate a maneuver would result ain brief reriods of 
nom-stationarity,y amd the model would rerform iinadeauately 
during and immediately after the transition reriod, 

It may be ardued that every rractical Frrocess can be 
considered stationary over infinite time. If the Prrocess is 
randoms it rerresents an ensemble of rossible rathss of which 
anew «6 realization im terms of real-world date is only one 
FOossible rathy said mays or may not be closely rerresentative 
of the ensemble. When dealing with reality, we sare often 
forced to assume stationsrity in order to make analysis 
FOSsibley and often we obtain good results even though we can 
mever kmow whether or mot the assumetion of stationarity 15 


really valid. 


Ce. QUALITY OF THE TIME AND FREQUENCY DOMAINS 


1. Fourier Series 


A very wide class of mathematical functions may be 
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Pee resenicen me tne Fourier series Eref.121]1 es follows: 
go 
Sli) = 2. + 2s "O68 mt + be Sin mt? 


where O 2 t si 27. 

Existence and convergence of this series rerresentstion 
reauire only that 8¢t)d be everywhere sinsle-veluedy and 
POSSESS 38 finite number of maximsas minimaery and finite 
discontinuities, The Punetion s(t) need not he 
differentiable. Any funetion meetins the sbove criteria can 
be thousht of as a constant mean function 2g» Flus an 
infinite series of sines and cosines of intesral freeauencies 
and various amplitudes. Of course, the inderendernt variable 
t must be shifted and scaled to the interval € Ors27]. Note 
thet tne lowest frequency rreserrt-s aside from thie 
zero-freauencys means 165 one cycle for the sran of s¢t). 
Amons tne functions meeting the criteris gre 8 sauare Fulse, 
an Uilmrulsey and ans manifestation of @&8@ random walk. In 
erseacticey the Fourter analysis of 2 function s(t) reauilres 
the trumcation of the infinite Fourier series. This results 
In 8&8 smooth least-seuares a@rrroximation to the function s(t). 
There ere rirrles ain the arrroximation if the function s(t) 
16 mot differentiable or if the truncation is too severe, 
This 18 known as the Gibbs ehenomenony and is illustrated in 
Figure ly whicn was taken from Hamming Cref.7]. By taking 3 
sufficient rmumber of terms im the Fourier exransiony we car 


immerove the closeness of the garrroximaetion. 
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Figure 1. The Gibbs Fhenomenon 


2+ BKasic Comeert 
The bhaesic concert of the duality between the time snd 
freauencs domains 15 so Simrele that ait often gets lost in @ 
forest of Fourier transforms. The time Freriod is the 


reciprocal of freauencs, The besic relationshnlr 156 


v/in = f = LT 


Where Vv 1s the freauency in radians/unit timey f is tne 
freauency im cycles/unit times and T is the time reriod for 
one cyvole,. Stated simelysy, freauency 156 the inverse of the 


time reriod. 


3+ Discrete DTlatea and the Samelins Theorem 
The disiteal comruter allows the efficient analysis of 
continuous Fhenomens by means of discrete arrroximations,. We 
S$38W earlier that the lowest freauency contained in @ Fourier 
xPaension of &@ function #¢t) was the recirrocal of the time 
Sram covered by the function. Similarlyws the famous Samelins 
Theorem Crefs. 6 amd 7] states that if 3a function s(t) in 


continuous time is semrled at comstantys discrete time 


a 





imtervals At (that aisy at @ rate of I/Atd» then the hishest 
observable freauency 15 0.5 cycles rer messurement interval 
Kt. Tnis means thet at least two observations are reauired in 
each cycle im order to observe thet rarticuler freauency. 
Tne freauencys O.,5/Aty wsualls Written sSimrely O.Sy 1s 
referred to as the Nvyauist freeauencs. The result is the 
eliasing Fhenomenony which 1s familiar to most moviesoers, 
Yurins the chasey the stasecoach wheels aFrrear to stor or 
rotete slowly backwards when the rate of rotation of the 
wheel srokes (srokes/sec) exceeds 1/2 the camera rete 
(frames/sec). When hisher freauencies exist im the funetion 
q(t) samrled at a rate Aty they are folded back and B@erear in 
the freauencs srectrum of the sameled data as freruencies 
less than the Nevauist freruencs, The samerlinsg theorem shows 
that 3 srectral anmalyusis of discrete data is only meaninsful 


Over tne Nsyauist interval €C -O.3/Aty +0,.S5/At], 


4A, The liscrete Fourier Transform 

Any funetion 4€¢t) for which 2&8 convergent Fourier 
Series exists may he rerresented in the freauencys domain in 
terms of real and imaesinars rartsy or in terms of amrlitude 
and Fhase andgley as a function of freauency. It should he 
noted that the function #¢t) may also be complex-valueds but 
We will deal with onls real-valued functions, In the 
continuous domains the formulas 


} 
g(t) =e [ow exe Civt) dv 


and = 


ine 





OoS 
Gv) = face exeC-ivt) dt 
= 


rerresemt @ Fourier transform frair. The freauencs resronse 


Gov) commwletely determines the time function 4¢t) and 


comversely. 


ioe tne function a4¢t? is samrled at iumnmtervals t+ = 
Ovilvyexreeeen the time-to-freauencys transformation becomes 


a 
Giv) = D a(t) exe -ivt) t = Ovlyeseen 


tto 
which is defined omls om the Nyauist interval (-7T,WT]y here 


defined in r8sdians. The formuls may oe written in 28 more 


familiar form bs using the Ewler relation 
“xe C-Tvt) = cos vt - 1 sin vt 
3s 
fr 
Gv) = ? Str ymeccCos Yoes £ Siri vt) 
€=20 


Which is continuous in v on the interval (€-T% 7. The Fourier 
transform is a hit difficult to handle analytically for all 
fut the Simrlest functitomss hut the discrete Fourier 
transform is senerallyvy easy to comrute hy use of Ss Fast 
Fourier Transform (FFT) frodram available in most commuter 
libraries. The outeut will generally be a very close discrete 
aPreroximation to Gtv)d, if the sean of #¢(t) ais Large enoush. 


The inverse transformation cam slso be made, 
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Since Gv) is comrlex valued whenever the munca ton Ct) 
186 not summetrics it is often useful to rerresent it in terms 


of amrlitude and ehaese, The smelitude is 


locvof = /Glv)Gi(-v) = 


where Re cv) and Im¢v) are the resl and imesiners rarts of 


CRe(v) + Cimcv) 7 





Giv). The rhese angle is 


O6¢v) = aretan Imcv) / Rectyv) 


fee UCOR-MEYER-FISK DECOMPOSITION 


Ir most Fracticel arerlicationsy 3 finite-vseriance 
same le-continiwous stochestic rrocess X(t) csn be writter: 


a 7 


oles ACO) Ff [ovo i facoauer 


o 0 


wnere X(Q) is the initial value of that rrocessys ACt) dt is 
Fredictahbleys smooth behavior determined Dy set of 
deterministic differential eeRuations describins the systems 
and BRC(tddWCt) 15 moisery where dW(t) is white Gaussian moise» 
and R(t) is 2a smooth trensformation that is sometimes thought 
Of a8 “coloring’ the moise, Such @ rerresentation is called 
the YToob-Meyer-Fisk decomrosition Cref.S]» which may be 
thousnt of 8s seraratins the rrocess into @ signal and moise. 


several imeortant roints must be macie with regard to this 


20 


a. 





eauation. ie tao imtended here thst the esrression be 


aveliusted analytically. The intesrel Bie Git os rai. CLO 
witesgrsl » whicn Pence) so  SbLOCMSSLic version of 3s 
StieltwJes UriLeadt tee Teh +O a « Alsoy alLbthousgh the rrocesses 


ACt) and ECt) sre smooth functions thet may be considered 
daterministic rerresentstions of ssstem behevior»y they ere 
mot necessarily known to the observers even when an adeausate 
technoalosical rerresenmtsation exists, 

Comsider 2@8381m our Filoted sgircraft beins trecked bys @ 
radar. The Frocess aACt) rerresents the dynamics of the 
airframe, aes affected by the control aunreuts of the Filotys 
Which are unknown to the rédaer observer. The rFrocess BCt) 
consists of several rearts., One 15 the measurement rrocessys 
which més or mas nat he known to the reader observer. For 


G@xemnreley,y s mutetins reder antennae might imrose some rertodic 


Seror if tne messurementy which would be manifested im the 
Paces s Ett). Qverlsid om this misht be eae white Gaussisn 
MOLSe messurement error. Air turbulemce could also he 


rerresented 3s white Geussian moiser whichy howeversy could 
Only be manifested throush deterministic airrlane dynamics. 
There ere those who would ardue that the rilot should slso be 
modelled as a random variable. In anys events the frrocess 
R(t) misht be further decomrosed into several rrocessesy here 
8st least airframe resronse to sir turbulence end reriodic 
redar amterms dynamics. 

The vital observstion is that if the freauency contant of 
the erocesses ACt) and BCt) sre known to be differents thes 


Sam He Fsertislly sersrsted by 23 srectreal anelysis of the 


aoe 





Zabs. Im our exsamrley gireraft have naturel dynsmic resronse 
freauencies ain all control exes. Tnese can be estimated 
Closely», even for enemys sirrlanesy and sre senerslly similsr 
amons similar teres of airelsnesy althoush they vars with 
airsreed. [t 1S FPAysicslls aimeossihle for the Bgirrelane to 
resrond faster than its hishest matursel dunamic freauencies,. 
Any freauency content higher than this must be moise. If the 
radar system dynamics are of 2a hisher freauencs than this» 
they can 3slso he serarsted. The fFilot will tske sdvantase of 
the full resronse rate of the airrlane only very rearels. 
Therefores low freauency comronents are most likely due to 
ePLlot maneuvers. Of coursey since white Gaussian molse nas 
3 flat freauency srectrum aes 32 result of aliasing Cref.7]: it 
1S imrossihle to serarate all of the moise from the signal. 


Howevers it 1s often rossible to remove auite 3a hit of it. 


Peel Gt TAL FILTERS 


A disgauteal filter 1s 32 linear transformation arrlied 
iterstivels to s set of data eoints. The Furrose here is to 
Sersreata moise from the signal. The simelest disital filter 
1s the simrle averssgey which estimates the mean value from 
the dstay and smooths out all fluctuations. The most general 


Form of the disital filter was stated by Hamming Cref.71] as 


m= oo 
u(t) = Dak) aCt-k) + Booch) x(t-k) 
br-8 aa] 


wWwnere the estimate «(t) at some Froint t is 3 ILineer 


ie 





combhinstion of tne dats roints 2z¢t-k)d» and rerhars of the 
ryrevious estimates x«(t-k)d. The coefficients ak) and o¢h) 
are weishtinsg coefficients end mays, of coursey he zero. AS B 
result of the samelinsg theoremy the filtering frrocess 15 


meaningless wumless the measurements = (te?) are mae et, 
eausllysy srsced tmtervels slons the t axisy wnere +t LS 


Mmemetivs but mot mnecesssrilyvy, time. 
ie SOME Ll lssslrications of Risgitel Filters 

Yisitel filters mas be classified 35 symmetric or 
Mon-symmMmetricy and 35 recursive or mon-recursive. A 
Symmetric monm-recursive filter is one im which sll bCkK) eanusl 
meee ans 2gll eth) = e@(-k)s such as the filter 

PaeroememeOge =(Ct=-1) £ 0.6 2¢t) mon os Ct tt). 
Am examrele of 3 recursive filter is 


toe) 8 SC) a Ct — 1 Onatl, b = Ll-a 


which i165 mot symmetric. This rarticuler filter maéy be 


OxFe pressed 39 


Peepers Gb) +t BE a 2=Ct=-1) + Wl Ss ztt—-2) + .++edd 


mater reduces to 


rr 
Mit) = o@ w(t) + ash =zC(t-1) + Barer oy Eee tee Loi) bres 6 
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The recursive filter extends to the infinite Fast, slthoush 
the coefficients ab will arrroach zeroy if Pane Li. tind S 
Caeser 8 recursive filter can he closely @rrroximated by 3 
nom=recursive filter. A Frimary advantage of the recursive 
filter is thet old dats mead mot be stored. New estimates 
may be commuted simely and raereidly as time evolves. This is 


an imreortent sdvarntsse for real-time arrlications. 


2. ArFlications of Tisitsl Filters 

Cisgitsal filters are used to sersarate 3 signal from 
moises to serarate various freauencs comronents of 3 sismals 
and/or to rerform such mathematical functions 33s integration 
and differentiation. A review of Simesom’s rule and the 
Trearezoidal rule should comvince tne reader thet these 
mumerical iaiumtesration technieaues srey Mie eat ac ts recursive 
digital Pilters.— Sometimes a filter has two Furroses. For 
examreley it might he desirable» im astimating velocity from 
SUCCeSSLIVe observations of Frositiorny to simultaneously 
differentiate and remove hish freavency moise. When 3 filter 
1s wsed to stor rart of the freauency srectrumy ait 1s 
referred to as 3 "low-rass"y "Hish Fass", “band-rass"y or 


*"Haend-stor*® filters derendinsg on its function. 


So. Ansglysis of DUigitel Filters 
In the time domsiriy coum art sd filter is deseribed 


Comeletely by its imeulse resronse functioms which is nothins 
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more tnan the resronse of the Filter to data consisting of 8 
strings of zeros»y 2a sindle oney followed by zeros, The outrut 
of the filter is then simrly the weishting coefficients 3a(k). 
If the filter is recursive, we misht mot be ahle to deduce 
the recursive form from the coefficients a€kdy but that will 
mot concern Ws hNere. The Fourier transform of the 


imreulse-resronse funmetiors 


Hev) 


3a(k) exed-ivk) 


h=-N 


a al 
7% 


Will comeletely srecify the freauency resronse of the filter, 
If tne filter 15 syummetricy there will he no imastinery rart, 
and hence no rhase shift. If the filter is recursive, it 
cenmot Fractically he symmetricy amd tne summation will 
generally run from zero to infinity. Theat iss the imrulse 
resronse will extend infinitely far aunto the futuresr which 
means that the filter remembers all of the rast. 

The duality of the time and freauency domains allows us 
to srecify 3a desired freauencs resronse and to desisgm an 
BPeerorriate filter hy calculating filter weights» or to 
analyze an existins filter by calculating the freauvency 


resronse from the filter weishtinsg coefficients. 


F. DATA ANALYSIS ANT EXPERIMENT DESIGN 


No digital filter should he aeFrlied to date analysis 
WEtnoOUt 3 clear ides of the effect of the filter uron the 


Asgt3. Sluteky and Yule first moted that some smoothing 
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formulas induced reriodic functions ain tne smoothed estimate 
that were more the effect of smoothing than of the orisinal 
data Cref.7]. A srectral analysis of rerresentative raw detsa 
can he nelrful ain deciding on an oarrrorriate filterins 
techmniaue,  Howevers such date 38s economic time series or 
weatner date are tyrically very moisy,s are based on 32 
relstively short run of datas» and cannot be described bye an 
adeauate technological model. Tne analyst must be aware of 
these vrohlems. Sometimes there sre mo good solutionsy but 2 
S$rectral rerresentation mas rroduce freauencies thet can he 
exe leained on rational srourids. 

AnQther Fotentisal eitfall is 2 result of the samrelins 
theorem, Comsider the timely xamrele of an air Follution 
model. It would oe resonable to susrect that gir rollution 
WOild follow at least 32 daily ecvycley or Frerhars an eisht hour 
cycle aif morning and evenings rush hours were considered. 
Usily sameles of air rollution could not hore to uncover 
cycles of 8 shorter reriod than every two days. Samrles 
every four hours would he marsdinally adeauste, Hourly 
sam@les would he mecessary for a good analysis. Additionally,» 
recall the reauirement for eauslly-sraced sampling intervals. 
For various reasons» the analyst mas have mo conmtrol over 
deta collection. Howevery he must always understand what has 
heen donner or could have been donmey to the datey as well as 
whet he is doings to itr im order to avoid erroneous 


Some LUSiTLOMms + 
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ITI. THE LINEAR DISCRETE NALMAN FILTER 
A. DESCRIPTION 


The linear discrete Kalman filter 18 & recursive Bayesian 
least-sauares estimator of the state vector of a IJlinear 
sustem based oma vector of moisy messurements made sat, 
discrete time intervals. Tne frocess to be estimated is 
assumed to be gen m-state Gauss-Markov Frocess of order ky» 
suhbuect to rroacess moise W with zero mean and covariance 
matrix QQ, The Frocess 1s observed Oy 3n m-dimensional 
messurements  suoJect to measurement moise VY (not to be 
conmfused with freavency ¢(v)) with zero mean and covariance 
matrix RR. The filter reauires 32 fFrior Kayesian estimate of 
system state arid covariance. The recursive estimate of system 


state st time t is obtained by the formula: 


PeGtnecomeete Gate t KCL EZCE)-HXCtit-1) 2 


Where 
oat; t) state estimate based om current measurement 
met) current measurement 


XCtit-1) state estimate rrior to currant messurement 
Rot) Kalman S¢8inm matrix (to be discussed later) 


tH Oobservsetion matrixs, which is constant 


Tne derivation of the Kalman filter eaustions may be 
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found in Gelb Cref.4]. A Summarys of the filter eauations is 
eresented in Fisure 2» which should be consulted in order to 
follow the subsequent discussion. 

In sgenerals the state model rerresents 3 dynamic systems 
that 187 ome which changes witn time. The extraralation of 
the state estimate tao the time of the nmext observation is 


obteined bys the formule; 


oct =~) ALts ct) 


where G@ 1s the state transition matrix. The observation 


Zz=PTOocesSs occurs seccordins to the concertusal relation 


etme HACE} FV 


where X(t) is the true system state observed throush the 
Observation matrix Hy and V rerresents measurement moisey 
which is assumed to he @ Gaussian random variable with zero 
mean and cavariance matrix RR. Note that the Frocess 
rerresented by this formula is assumed to occur in the real 
world. The commutation does mot accur in the filter. Rather» 
the measurement Z(t) is an imreut to the filter, 

In the linear Kalmsn filtery the gain Kt) does mot 
derend in amy way on the dats. It derends anly on the model» 
and is therefore extremely sensitive to assumptions. Gein is 
esleulsted according to the formula 


K(b) = PCLOH CHPCLOH + RI 
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where FCt) Ls tne covariance in tne system stste estimate 
-rior to the current messurement end K is the coveriance of 
the messurement errors The coveriance 1s vurerdseted sccordins 


to the formule 


See = ONC GOHI P< t) 


Bere 2.¢(+t) 1s the stete covariance given the current 
measurements and I is the tdentity matrix. The covariance is 
extrarolsated to the time tmmedisately rrior to the next 


observation boy the formuls 


a 
P(t+i) = BS(t) e+ a 


mmere @ 15 the state trensition matrix and @ is the 
covariance of the Frocess moise. Combining the sbove two 
eaustions shows tnat tne covariance of the state estimate st 
the time of the current measurement derends on the fFrevious 


coverisnce accordins to the formula 
r 
Ze eaten to HI Co 2¢t-1>) 6 + QI 


Filter rFrerformance LS vers derendent on  aedeauate 
modelling, earticularly on the state transition model and 
the choice of moise covsriances KR and Q. To a lasser extenty 
rerformance e@lso derends om the initial estimates of system 
state X(1:90) and covariance FPC1i0), Howeversy the latter 


raraneters ere less imeortent because their effects decresse 


30 





With time. If the matrices Ry Qy Ge amd H are comstant ir 
meme tne ssaim KCt) emd covarismce matrices S(t) and FCt) 
eventually reach 3a steady States sand are comrletely 
determined Oy Ry Qy Gs» and HH. 

For @ sdivenm linear filtery it will be shown that filter 
ging coverisncey and freauenmcy resronse will be comrletely 


determined hy the choice of RK and Q. 


Poe tHe SCALAR KALMAN FILTER 


The multi-state Kalman filter is @ Fowerful comrutstional 
device, However, itis difficult and often imreossible to 
manifeulate im closed form because af the freauent cecurence 
of singular matrices. An analysis of 8&8 Simsle-state (scalar) 
filter can be used to illustrate the mechanics of the Kalman 
filters and to sid in sdeveloringsg am intuitive understandins. 
In the discussion that followss it is assumed tnoat all 
matrices are scalars,» andy im rarticulsery QQ and H eaual one. 
Matrix motsation is rreserved for clarity. Derivations may he 


found im @rrendix A. 


ie. Transient and Steady-State Gain 
[It can be shown Carrendix A) that the scalar Kalman 


3in cam be exrressed recursively as 


R(t) = NC eee 1 
Poe sete ti TN. oF 1 


When the filter reaches steady-state, tne s31M is constant 


a 





aricd 


K = -a + /_@*+ Q 
oR 4? Fe 


The inverse relationmshir is 


milo 
i 
es 


Thusy the varisnce retio Q/Rs, which is the ratio of rrocess 
moLse varience to measurement moise variances completely 


determines the steady-state s3in. The steady-state filter is 


comme letely described hs the formuls 
MOCome IN Coto + CLSeR) AC t=-1) 


2. Frequency Resronse 
Lettings K = 3g and ¢Ci-K) cay y the imreulse-resronse 


Furmetion Gt) may be written 


t 
G¢(t) = ab ¥ tm Og Weis. +o 6 6 
The Fourier Transform is 
Gm 


Hiv) = Joc exe CH-ivtddt 
- 


oD 


c 
Htv) = 8 7 tb exe C-1V) 7 
t=o 


Héev) = @ / Cl - b expr (-ivdd 


Be 





Since the fel Loe ls not symmetric» the freaquency reSrorse 
Hiv) hes hoth reel end imesinaerys rarts. The amrelitude mas he 


wWritteri 


A= jHtvo| = fdtwoHt-v) = a7 fi + b*-2b cos ¥ 


which reduces to 
A = Q/R 
Gives ok cos vy?) 


The rhaese angle may te written 


O8¢v) = erecta ( “hh sir Vv 
1 -—h cos v 


The angle for maximum rhese shift 15 
Vase). = areeas © = srecos (I-h) 
2 2 
mecmax OF = arceas (1 ¢ Q/2R - Qo/4R 0 + ASR) 


Therefore, the variance ratio Q/R also comeletely srecifies 
the steady-state freauency resronse of the filter. Amrlitude 
and rhase relationshir for several values of gain are rlotted 
in figures 3 and 4. 

It is evident that Aish Q/RK Chisgh sain) reduces the fFhase 
las of the filter but sllows more of the hish-freavency 
Comranenmts ta FBSS + Conversely low Q/K (low sa1Mm) 
attenuates more of the hish-freauency comronentsy sat the 


exrenise of an incressed rhaese led. Note thet even at vers 
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FREQUCNCYT RESFOMSE OF SCALAR KALMAN FILTER 


AMPLITUDE 
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low gaim Clow Q/R) mot all of the hish-freauency comronent 15 
attenuated, snd the Fnase les is auite severe, The slore of 
the amelitude chanse 18 auite shallows melying that 
attenuetion increases gradually as freaguencs incresses., This 
15 32 conseauence of the assumetions and rarformance will mot. 
he adeauate aif the data does mot rerreseant 3 Geuss-Markov 
Frocessy hut im fact rerresernts some Fhenomensa changing with 


Lime. 
(eis IMPROVING THE FILTER 

The transition band of the filter can be sherreneds and 
more of tne hish- freauency comronernts eliminateds bs runnins 
tne date throwush two filters im series, The basic scaler 
filter was 


SOG mens 2 CT) op ia eC E11) 


wnere 3 = K and b = C1-K), Runmoins the data throush the 


Mmeeber S8scirirs We obtain 38 mew estimate w(t) + where 


Saves ae GG) of fm wt t= 2) 


meesnould pe evident that we can accomplish this all in one 


SLa- 325 


2 2 
w(t) = 8 z(t) + 2h vwtt-1) - b ¥vCt-2) 
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We meed onmlws to save one additional rrevious estimate wlt-2) 


3s Well es wlt-L). The imrulse resronse fiunctiom 15 


a(t) = (ttida®b  » t = Ovteses 


We heave rerformed 2 comvolution in the time domain: which 


corresronds to a2 multitelication iain the freauency domain. 
This may mot be exactly what we want. Let us surFrose that we 
Want the weishtinsg coefficient for the fFresent data Froint 
z(t) to he 0,27 in both cases. This reauires 8 = O.27 for 
Memiissic filter amd @ = fO.27 = 0.52 for the double filter. 


we 


The imeulse resronse function for toth filters 1s Fresented 


cr 


in figure 3S. Note that the douhle filter forgets the rast 
more readily. The amelitude and rFhase shift for both filters 
1s Fresented ain fisgures 46 and 7. The gain for the scalar 
Filter was 0.27% corresronding to 8 variance ratio (Q/R) af 
O.1. Also mote that the double filter has somewhat better 
Hish-freauency attenuationy somewhat less attenuation at low 
freauenciesy and a slightly sharrer (steerer) transition. We 
WOuld therefore exrect it to he 2a bit better st serarating 2 
low-frequency sisgmal from moise. At low freauenciesy the 


momole filter has less phase snift. However» wilesce —ShiliG is 


more severe at freauencies sbove 0.5 radians. 


NY. FERFORMANCE COMPARISON 


The basic scalar filter snd the imrroved (double) filter 
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Were comrsred wsins the data Mewar rslLearlm Taigure &. ra 


series of imdeaerendent mormal random numbers with wero mesn 


ama 6|CUVEePLance ten were senerstedys to simulate measurement 


ieee. [nes Were egdded to 2 nominaegl function which was 


G3 


combination of sters and 3 rame. The rame function rises 1 
unit each measurement aumtervalys which corresronds to one 
standard deviation im rrocess moise Q»y so at least during the 
rame funetions the data corresronds to the filter design 
Variance ratio Q/R = 0.1. A srectral analysis of the dats 15 
eresented im fisure 9%. The signmeal and moise are fFresented 


Sserarsately and im combination. Note that the moise-only scale 


1s exranded. The mo1se srectrium 1s lrresulary but overall 
Garte flat. The signal consists mostly of Very low 


freauenciess but slso hes some hish freauencies. This would 
be expected» since ster snd rame functions reauire very high 
freauencies im their Fourier exransiorn. The nish-freauencsys 
Ssisgn3alo1is submerged im moise. The datayr of courses does not 
fulfil the assumetions from which the Kalman filter 1s 
derived, Howeverysy the resl world seldom does either. We sre 
looking for robustness. 

The filters were first tested on the si¢mal slone. The 
results ere rresanted im fisure 10, Tt cam be seen thet 
meither filter Car resrand instantaneously 3G the 
discomtinuities im the functiony since Hhish freauerncies are 
attenuated. Roth filters las after discontinuities and 
suring the ramr rise, This 15 2& conseauence of the non- 
Summetric mature of the filters and aillustrates the Frhese 


lad, Note that the double filter rerforms 3 bit better» 
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remaining closer to the date throughout (1t is 32 reculigrity 
of the Flottingd routine that onls one symbol will be fFlotted 
wnerm ever dats roints coincide. Tnereforey date roints thet 
do mot arresr should be resgarded as occurring simulteneouslsy 
with the ones that do arrear), 

The filters were then tested oan the moisy data. The 
results are frresented in fidures 11 and 12. The latter Flot 
hes the data surrressed so that the scale can he increased 
and more resolution obteined, 

The same trends can be ohserved as were Freviously. The 
double filter lags less during the trend and transitions. 
The double filter arrears toa fallow the moise 3 hit more 
Closely, but overall it follows the sismal better than tine 
bhesic filter. The average variance between the sidnal and the 
filter was 4.20 for the double filter and 3.10 for the basic 
filter» which was an 1924 imzrovement for this simele 
modification. The imrrovement is due to the fact thet the 
double filter weights more recent data more heavily: ana 
remembers less of the rast then the scalar Kalman filter» 
even thoush the weisht on the feresemt observation is the 
same. 

This simele exreriment is only intended to acausaint the 
resder with rossible imrrovements to the Kalman filter. Like 
amy tools the Kalman filter should mot be arrlied 
indiscriminantlsys. The interested reader is referred to 
Hsamins Cref.71 for the basic errincirles of digital filter 


dasismn. 
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Pree ihe TRANSIENT CASE 


The sorhisticsated reader has mo doubt moticed that tne 
steadu-stete scalar Kalman filter 15 eauivalent to the 
Box-Jenkins IMACOv1s1) model Cref.11] and to Rrown’ s 
exponential smoothing model Cref.10]., Zehnma criticized the 
exroneantial smoothinas model Cref.13]+ notins the odias would 
Seeur if the steadu-state model was arrlied With an 
Imna@rPrrorriate Frrior estimate, Ressler and Zehna Cref.14] 
develored 2a 31m schedule which they call finite exronential 


SMmMOOtLKLInNS > fmeir 'TOrmule Tor S$sin 1s 


a(t) = a/(i-b ) 


where e is the stesdyu state sgain»y b = Ci-a)» and att) is the 
gain schedule as a function of time. It 18 Similar to the 
Kalman sain schedule aif the initial Kalman gain KOO) 1s 
enosen 85 one. In tooth models»s ma Frior estimate 16 
required, The weisht on the first observation 1s one. A 
comrarisonm of the two models is illustrated in figure 13% for 
@ steady-state sain of 0.2 and an initial sain of 1. Tie 
Kalman sain was calculated sccording to the recursive formuls 
im section III.A. 


The Kalman filter sain converses fastery althoush the 


13 


difference is mot great. The scalar Kalman filter rossesses 
two other advantases over the finite exronential smootnins 


techniaue. Firsts if 28 s#00d Prior estimste does exists: the 
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Kalman fllter allows the initisl gain to He chosen es less 
than omnes and its value will be determined Oy the assumed 
covariance of the frrior estimate. Also,» tne Kalman model 
forces the analyst to at least think shout the conmcerts of 
measurement noise and rFrocess moisey and to estimate the 
moise variance ratio Q/K. 

As moted bheforey 2 freauency analysis of the transient 
case is mot arrrorriate. Howevery it can be thousht of 3s a 
case of transient handwidth. The filter is initially set to ~2 
hisher sain then stesdy-state sain. If there is mo rrior 
estimate available, the initial gain 15 ones and the filter 
is initially an infinite-handwidth or all-rass filter. As 
dats ere acauired gain drors ard the handwidth maerrows until 
Steady-state conditions are achieved. The concert of 
transient bandwidth 16 aimroertant to the subvect of adartive 


filterimss to which we will return. 


Peet LGHER ORDER FILTERS 


The main beauty of the Kalman filter 1s mot in its 
statistically unbiased method of calculating gains But im its 
Fowerful matrix formulstions which allows it to he arrlied as 
2 multi-dimensional model iaimeorrorating anv order of 
differencins desired. As the state srace 15 inmcreasedy it 
Quickly becomes imrossible LO analyze tie filter 
analytically. Hisgh-order multi-dimension filters can also 
essilw exceed the caracity of Fresent digital comruters for 


re3l-time ea@rrlications. Fortiumatelyy it hss been found that 
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the state sraece can oe reduced and dimensions decourled with 
vers little desdradation in the overall asccuracy of the state 
estimate CLrefs. 4 and SJ. For exameley if 3 12-state model 
cam be reduced to 9 states and can be sadeaustely rerresanted 
hy three 3-stsete modelsy the matrix calculations can be 
comsiderandoly simrelified and sreeded ur. 

We will examine 3 second-order (first difference) 
Filter, which can be used to estimate trends or velocity. We 
Will use the latter term. Fosition and velocity sre to te 
estimated based only on successive measurements of Fosition, 


fees ctecte transition end tne observation matrices ar 


iD 


ast 
Cae 


ti 
il 


a 
il 


bide 


The covariance matrices Zs Fy and Q ares of coursery 2 by 
2 matrices. The state vector has two elements», velocity and 
PFOSitions while the measurement vector has only Frosition. 
The measurement error RF is @tl ay tL matrix which we will 


V3ry. Wea have chosen Q 32s 


Q= |0 0 
Oa 


ersuins that anew rrocess moise will be conmtained entirely im 


velocity. iinstetss tnere Can ne mo random motion thet is mot 


caused by 3 rendom velocity. Randomness of velocity will feed 


imto rosition through the state transition matrix. 


Dee 





Even ain this simele casey solvinsd analytically for 
StLesadu-state s3in ain terms of R and Q reauires solvins 32 
system of 4tnm order Frolsnomial eaustions. We will ort 
instead for 8 commuter solution. Tne reader may continue to 
think in terms of the moise variance ratio»ry where KR will take 
om the values Ly 10% 100 and Q will remain constant as sbove. 
Since there 1s omlye one mom-zero term im the Q matrixy we may 
think of the moi1se vsrience ratio 3as the scaler auantity 


meses 2) /R-. The resulting steady-state sdsins are 


Noise Variance Fosition Velocity 
Ratio Geir Gain 
1.90 +759 »481 
O.1 cod eee 
0.901 TOO »O80 


As would be exrecteds the rosition s3a8im 16 much hisher 
than that of a scalar filter at an eauivalent moise variance 
ratioy because the Frocess variation mow serlies to velocity 
rethner than rosition. The velocity gain 15 comsideradbly less 
than the rosition Sains since the velocity 15 not measured 
directly Out must be estimated from successive measurements 
of rFosition. The imPeulse-resronse function of the medium-sain 
Filter (noise variance ratio 0.1) 16 Ffresented in fisure 14. 

The amelitude resronse of these three filters 1s 
comeared im fPisgures 15 and 164. The most striking feature 15 
the amrelification which occurs at 38 srecific freauencys im tne 


FOSition frequency resronse. This imrelies that the filter 1s 
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FREQUENCT RESFORSE OF VELOCITY ESTIMATE 
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most sensitive to motion in @ rarticular freauency ranse,. 
Thusy tne mstursl freauerncs of the system to he observed, if 
1t 18 knowns 15 8 sidnmificant design rarameter. ASsiriy we 
observe the settenuetion of hisgh-freauency moisey altnoush 3 
Ssi1amnificsent amount still remains at the maximum freauency 
(note that fisure 15 does mot imclude the orisim). 

The freauencs resronse of velocity shows 3 reduction in 
ame litude 3st low frequency. The amrlitude at zero freauericy 
16 eeuel to the velocitss gain, This 1s far from ideal 
rerformance for 28 <differentistory, which should have an 
ame litude resrornse of zero at zero freauency, with 3 slore of 
one we to the cutoff freauerncys Cref.7J. The differeritiator 
1sy Howevery ressonahly effective at reducing the amelitude 
of hish-freauency comronents. 

The ehase shift of the filters 38881m shows increasing 
Fnaese lest es sein 15 decreased. Tne overall effect 16 
Slmllar to the scalar filtery ‘and is otherwise unremarkable. 
Thereforey Frlots gre not included. 

The data of fidure 8 was tested on the lowest-dain 
YV@locitysy filter. Note that the seectral content of the data 
(fidure 9) is auite low-frequency, and that the bandwidth of 
the lowest sain velocity filter is auite Wides and indeed 15 
higher then that of our scalar Til cen. So it might he 

“rected that the velocity filter would have some trouble 
With the dsta. 

The velocity filter srerformance on the nominal funetion 

only is Presented in figure 17. The filter overshoots auite 


badle st the discontinuous stersy Whichy of colursey are ar 
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imPeulse im velocity. Tne overshoot is less at the start and 
stor of the trend. It does settle down and track the trend 
Without lass which 15 an imrerovement in rerformance aver tne 
scaler filter. It should be noted tnat with hisher gain» the 
filter would track the mominal data bettery while with lower 
gains the overshoots would be more severe, 

The rerformance of the velocity filter on the data is 
illustrated in fisure 18. AS expected» the filter tends to 
follow the moise too much. Howevery it does follow the 
discontinuities much more auickly than the scalar filter. 
This roints out the fact that the hisher-order filter is more 
effective 32s 2 maneuver detector out it is less suitable for 
smoothing very moisy date. Tnis asain illustrates the 
concert of bandwidth», which is auite nish even in tne 
low-s3ain velocity filter. 

In retrosrects the decision to choose Q(1s1) as zero may 
mot nave heen wise. Allowins some FProcess moise im Fositionys 
exclusive of velocity, could well have some smoothing effeet 
om the velocity estimates which would result in smootner 
one-reriod ahead rredictions. This could smooth tire 
Oreration of the filter 32 bit. The rossible combinations of 
filter rarametersy even for this simple filters are euite 
MuIMe rows. 

The Freauencs resrorse of 3 second-differerce 
(acceleration) filter was also determined for comrarison. 
Tne results are rresented in figure 19, The Q matrix was 
zero excert for Q¢3%3)¥ which was one. R was chosen as 10» 


resulting in 3 mominel noise variance ratio of OQ.1. ASSiLfy 
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the saim was nisher then thet of s velocity filter with an 
eauivelent noise variance ratio, The amrelification of 
low-freauency comeonents of Fosition was increased» and the 
zero-freauency amrelitudes of velocity and acceleration e@dain 


corresronded to filter Sain. 
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FREQUENCY RESFOIMSE OF ACCELERATION FILTER 
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IV. ESTIMATIONy SMOOTHING, AND PREDICTION 


A. ESTIMATION 


So fary we have been concerned only with estimation of 
the rresent stste. A filter desismed to Frrovide such an 
estimate cannot be syummetricy because it can Fut mo weight om 
future observations. Thusy rhase last 15 imevitahle,s and is 


one of the rarameters tinat should he considered in the design 


Popo Ot S « 


Poe SMOOTHING 


osmoothins is the use of a filter to Frovide an estimate 

Of rast states. Such 2a filter can he made symmetricy which 

comPletelys eliminates the rhase las. Nom-recursive smoothins 

filters cause 2 loss of N data Foints at each end of the 
datey where the sran of the filter is (-Ny NN), 

The Kalman filter cam be used as &@ smootner by simely 

running the forward estimate throush the filter in the 


Orrosite direction. The imreulse resronse function of the 


scalar filter was 


feces be shown (3rrendix A) that tne imeulse resronse 





Funetion of tne smootner GEaitdocrta welll  wWaeckWwerdg filters 
combined) 16 


act y= OR 


WHicn is Just the conmvolution 


a(t) @ a (-t) 


The Kalmen filter is able to rrovide an estimate 
thnrowshout the sran of the data. No data is lost at either 
end. However» due to transient effects, the data mear eitner 
end 1s subvect to rhase shift and some increase im Sain. The 
filter is necessarily not summetric mear each end of the data 
SrBrM.+ 

Gelb Cref.4] ineludes 3 comrlete discussion of 
fixed-roints fixed-lad, and fixed-interval smoothing, We 
Will restrict our attention to the scalars fixed-interval » 
Steady-state casey isgmoring the end effects. 

The scalar Kalman filter of section TFIII.0 (noise 
variance ratio of O.l» gain of 9.27) was used as 8 smoother 
om tne dats of fisure 8, The results are rresented in 
fidure 20. AS comrared to the one-rass rerformance as 
illustrated in figure 12» the smoothed data shows rhase las 
removed and reaks in the oscillations reduced. WHowevers the 
smoother nas less ability to follow tne discantinuities in 
the nominal funetion. The removal of the rhase lag is 


chneracteristic of any symmetric filter. Howeverry the reduced 
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ability to follow the discontinuities is the result of 
reduced effective Sain. since we have convolved tine filter 
weignts»s we neve seusred the amelitude of the freauency 
resronse. The we1sht on the data Ffoint at (t=0) is reduced, 
The effective gain was 90.27 for the forward filters and 0.156 
for the smoothing filter. This reduction im effective gain 
1s mot addressed in the literature on the Kalman filtery and 


it is wnelesar how smoother sain should be chosen im relation 


to the moise variance ratio Q/RK. 


CC. A COMPARISON OF TWO SMOOTHERS 


The Kalman smoother of the Frevious section was comrsared 
With 3 Geussian smoother to illustrate some design ortions 
and rrocedures. The Gaussian smoother was chosen from amons 
a huse variety of data windows because 1t hes sood smoothins 
Frorertiesy and hecause it is rearticularly easy to desian,. 
Interestingly» Freliminars exreriments Showed reretitive 
arrlications of ae Kalman filter to result in an a@rrroximately 
Gaussian filter weight distribution. A comrarison of the 
Gaussian smoother to 2 variety of other windows is contained 
im Harris Cref.1i5]. 

The Geussian smoother 18 2a symmetric filter with the 
Weights chosen according to @ discretized and truncated 


normal distribution. The formula 16 


2 
a(t) = K Geta 


65 





Mmarere t+ 1S 3m inteser on the ramse (-Ny»y N) emd KK 1s chosen 


such thet 


S a(t) = 1 


The esse of design comes from the observation that thie 
Fourier transform of the continuous Normal distribution is 
glseo 32 Normal distribution with scsle rarameter (vserisence) 

2 : . 
eaual to 1/6, As looms 325s (ao = 2) and trumcstion 168 not more 
severe than |N] = 26% 8a reasonable srrroximation of the 
freauency resronse for tne Gaussian disgitel filter tus 


a 


GOVIN™ exp (-o' Vy" /2) 


The scale rarameter was chosen such that the freauency 
resronmse was eauel sat (v = 0.3). Skirerins the alsearsic 
Hetails» this reauired (@ = 3.11). The Geussian smoother was 
trumcated to (N = 7)» resulting im @ filter sean of 15 data 
Fonts. The frequency resronse of hoth filters is rresented 
in figure 21» and the filter weighting coefficients are 
Presented in fisure 22, 

Simee we trunceted the Gaussian smoothers we would 
exrect some firrles in the tail of the freauency resronsey 
Which sre Just tharely visible in fidure 21. The Gaussian 
filter has a sharrer transition hands and is auite effective 
in Hlocking high freauvuencies. As commared to the Kalman 


smoothersy the Gaussian filter weishts the rresent data roint 
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FREQUENCY RESFONSE OF SINGLE AND DOUFLE FILTERS 
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lassy mearby dats Froints more end farther data roints less. 
The rerformence of the Gaussian smoother om the data of 


% 


figure 8135 Fresented im fisdure 23. A comrarison of the 
Kalman and Gaussian smoothers 18 Fresented in fisures 24 and 
2oe In fisure 24% the smootners are grrlied only to the 
nominal funetion. [It can be seen that the Gaussian smoother 
followed the discontinuities and corners of tne mominal 
function hetter than the Kalman smoother. Howevery when the 
smoothers were ~arrlied to the moiss datar tne results were 
less clear (figure 25). Tne Gaussian smoother assin followed 
the nominal funetion 38 bit betters but it slso followed 
low-freaquenecy comronents of the moise 28 Hit morey tendins to 
emrhasize cyclic effects that aren’t really there. The meari 
square difference hetween the smoothed estimste and the 
momingl function were very similary 1,81 for the Gaussian 
smoother and 1.84 for the Kalman smootner. Thuss tne Kalman 
filter seems auite effective when used as 8 smoother. The 
reader 16 reminded that the mean sauare difference for tie 
scalar Kalman filter was S.lvy which clearly indicates the 


Ssureriority of smoothing over filterins. 


ieee FRERICTION 


3 


BeCGahevlom 15 difficult. Recall thet 38 stochssti 
Process 16 an ensemble of rossihle rathsy while date is the 
manifestation of one member of that ensemble. What could 

eve herrened didy hut wnat can harren isn’t mecessarilsys 


so1ms to. Frediction can he thought of as filtering without 
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measurementss and toe estimate 16s fFroJected forward ir time 
through the state transition model. Filtered or smoothed 
estimates may be Quite sccuratey even if the state transition 
model 4s not. Howevers sSood Frredictions are heavily 
Herendent om an securete state transition model. 

The hnisher-order Kslman filters are Frolynomial models. 
Hammins nes rointed out that Frolynomial models are Foor 
eredictorsy since the estimate tends to veer off to Flus or 
minus infinity 38s soon @s the model is released from the data 
Lref.7]. There 18 mo reauirement to use the same model for 
rerediction as for filtering. For exameles it might make sense 
to track 3 target with an acceleration filters but to commute 
fire control information based on 3 constant-velocity model» 
Since tarsdet acceleration is generally sssumed random with 
Zero meare Similarly» the economist may desire to filter 
data with 3a@ nish-order modely Out make Frredictions based on 
constant trend. Clark Cref.SJ discusses 2 somewhat more 
SOPhisticated method due to Sinsers im which the model decays 
from an acceleration Frredictor to a constant-velocity 
eredictor 35 rrediction time increases. Such techniaues are 
heuristic im nmaturey but can rrove valuable to the immovative 
analyst, 

An interesting exemrle of the sebove concert can be found 
in Box and Jenkins Cref.11]. They compared 2 auadratic 
forecast due to Brown Cref.10] with thetr own IMACOv191) 
model with 3 gain of 0.9%, The latter model is eauivalent to 
the steady-state scalar Kalman filter. The date used for the 


Comrarisonm was 3 time series of IBM stock rrices, Box and 


ce, 





Jenkins observed thsty wnile the auadratic model might well 
ne used to fit the dats» its Ferformance 35 8 Fredictor was 
clearly inferior to the simeler IMACOrs1+1) model. its is 
mot Surrerisings since it has lons heen sussested that stock. 
Frices behave as a random walky and that the hest forecast of 
stock rricey at least im the short runs is the rresent frrice 
Peers. tid. Note that the foregoing imelies that tne sain 
should be set to 1.9% which corresronds to no filtering 3st 
Bll. Thnereforey Kox and Jenkins e8rraerently found that some 
filtering of the dats was a@rrror-riatery even though the sain 


they wsed was atuite Nish. 
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Vee SUE nem I NEMENTSOs EXTENSIONS AND ALTERNATIVES 


A. ADAPTIVE FILTERING 


In the linear Kalman filters the sain 15 comrletely 
inderendent of the data. Clearlys this will result in maJor 
errors if the sain is chosen inarrrorriately or if the deta 
statistics chamse. If the sain is too lows tne filter lags 
hadly, Im the extreme caser which occurs aif the Frocess 
moise covariance Q 18 much too lows tne filter rays much too 
much attention to the rast and diverges from the dats. On 
tne other hands if the sain is too high» the filter rays too 
much attention to the data and the state estimate contains 
noise. If the filter is 8 rolynomisal model and is to be wsed 
8S 8 Fredictorsy the resulting errors will be srectacular. 

The solution 1S simele in concert hut can be difficult 
to imelement. Qne simely sets the steady-state gain as Law 
8S SPrrorriate for the stahle rrocess heing estimated. In 
target tracking: the sain would he set to track an airrelane 
figinsg 2@ streisght rath. A “maneuver detector" or “trend 
detector*® 1s incorrorateds, which i8 motnins more than 32 
recursive statistical test arrelied to the residuals to 
determine whether or mot they come from 3 Zero-mesn 
distrifbution. If noty the bandwidth is gradually widened 
(gain 18 increased) until the residuals rass the zero-mean 
test. Thens the gain is allowed to decrease toward the 


stahnles steady-state value. 


ie 





Further details end some movel arrroaaches are discussed 
mame clark Cref.Sl. Two exsmreles tsken from Clark are 
tllustrated in fisgure 25. The concertual adertive filter 
discussed ahove reauires time to detect the maneuver or 
trendy adart to ity and reconversge to @ stable gain setting. 
Turing this times the state eastimate is less scecuratey snd 
the time reauired may be uwmacecertably longs for some 
srrelications. 

Clark rFroroses 32 dual-handwidth adartive filter to sreed 
sdaretetion. The Froces i$ Simultaneously tracked by 2 
marrow-band and a wide-hand filter. If S maneuver or trend 
1s detected, the state estimate of the wide-band filter is 
fed into the marrow-band filter. Ideally, this would allow 
the nerrow-band filter to JSumre immediately to tne current 
(unbissed) estimate of the wide-band filter. In Practices 
Clark found that some widening of the hanmdwidth of the 
narrow-dpand filter was also reauired, 

Voluminows literature exists on the subJects much of it 
vers difficult to read. Clark Cref.S] ancorrorates 2 
rarticularls lucid account of stability rroblems encounteredys 
methods of reducins the cost of false detection of bias: 
analytical methods of determining filter Frarameterss and 
exreriment3l results. Althoush Clark ’s filter was desisned 
to track and Predict the Frosition of airborne tarsetss, the 
methods discussed sre a@dartable to the aieer ins of economic 


time series or virtuslly any otner stochastic rrocess-. 
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KR. NON-LINEAR FILTERING 


In the nmonm-linear Kalman filters ome or more of the 
matrices Q» RK» Hy or $§ are sllowed to vary with time. Since 
this results aim 3 time-varistion of the sain matrixy 
quantitative analysis aim the freavuency domain 15 mo Longer 
aerrorriate. Howevers it 15 well to keer the concerts in mind 
im order to 481m Bdditionsal aimsisht. There are two basic 
LYPeS of mon-Llinearities tnet may e8Trisey non-linesr 


measurements and mon-Linear dynamics. 


1. Non-Linear Measurements 

Nom-linesr messurements arise when observetions are 
made in ome coordinate system and the model reauires that the 
state he estimated in another coordinate system. In this 
casey the matrices R and H are time-varying functions of the 
Meeainste transformations and do derend on the dates, im the 
sense that they derend on the location of the data within the 
coordinate system. This tyre of mon-linesrity is often easy 
to handle. 

For the best examele of non-linear measurements we must 
return to the target-trackinsg model. Fire comtrol systems 
generally track ain @zimuthe elevations and range. Howevery 
the model is 2 Folynomial in Cartesian coordinates, but mot 
Im rFoler coordinates. Airelanes often fly a straight rathys 
but seldoms aif evers fly 38 constant bearings or ranse with 


resrect to tone radar observer.: iti cnis models tne 
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mon-Linearity can be reduced by considering the Cartesian 
measurement error 25 8 Linear transformation of the roller 
measurement error. If the Foler measurement error 16 
Gaussieny the Cartesian measurement error is very nearly 
Gaussian with coveriance Meee lok) atk a function of the 
coordinate transformation. 

The Cartesian FR matrix will mot he diasonalys even if the 
FOlar RK matrix 18. Howevers Clark Cref.3J has found thet 
setting the off-diagonal terms of the Cartesian RK matrix to 
zero did mot @rrreciahbly degrade filter rerformance. In this 
Ways he was able to decourle a nmine-state filter into three 
three-state filters, 


If the measurement mom-linesarity is too severey it may 


mot he ressonsble to sssume that the moise is Gaussian¢ni.- 
However y» limited “rPeriments rarformed om data with 
non-Gaussisan moise (an exronential distribution was wsed) 


showed that the Kalman smoother and the Gaussian smoother 
were Quite robust es longs as the Ssin was mot high. This 
seems to he 8 conseauence of the Central Limit Theorems since 
low 4ain imelies 2 linear combination of a fairly large 
number of data roints, It should be noted that a filter 
designed to handle this situation is still Linear» althoush 


tne Gaussian sssumrPtion is violsted. 


vs Non-Linear Denamics 
Nom-lLlinear dynamics sre considerably harder to handle 
than mon-Linesar measurements. This is unfortunsatey since the 


are3as of Frotenti3al @rrlicsation ere mumerous. Nom~-linear 
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dynamics occur when the QQ or G@ matrices derend om the 
Previous history of the frocess,. 


As 8&8 simrle exsmrleys comsider the multirle resression 


y(t) = g a + b “Ct ) test tl) 


where uy(t-1l) corresronds to the iauntercert term. In this 
modely we wish to estimate the derendent variable y(t). To do 
SOrv we need to estimate mot only the inderendent variables 
aCt) and “(tds Hut also the redression coefficients @ and b. 
Let us assume we can measure ull» x (tds and Met)» but mot 2 
ang.) CO ASSUMInsg 8 First-order syustemy the state urdate 


eaustion 1¢S 


Geto ©.) Peetu) mem tt) OO SCC 1 Gp 
a a a 0 3. 0 Oo oO Ct it) 
CttLit) | = 0 0 if oO mbit) 
Be Ur 1) 0 0 0 70 aie) 
te ce QO Q O Oe! iC tein Ea) 


Where the transition matrix is unfortunately mot uniaue,. The 
first row of the transition matrix could tbe eaually well 


rerresented Oy 


Fi oO 9 xCtitd Sera) A 
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andj it obviously changes st every iteration. It is at this 
FOint that filter desidgm becomes an art. 

Note thet the inderendent varishbles and the resression 
coefficients are essumed here to he first-order Gauss-Markov 
FTOCCSSAS5-+ Increasingly high orders would multielys the state 
SFaCe. 

Several exreriments were rum wusing 38 second-order model 
Similer to the shove on the Box-Jenkins Cref.11] series M 
gata (ssles dats with leading aumdicator). Quantitative 
results gre not Fresented, because the Box-Jenkins data did 
mot inelude sufficient forecast estimates for comrarisonrs 
some “cheating” was done becsuse the Kox-Jenkins rarameters 
were wsed in filter desidgmy and it never became clear exactly 
Wnet earameters Were Berrrorriate for the R and QQ matrices, 
However? some aualitaetive comments are *arrrorriste, The 
model did work, Some instability was moted in the resression 
Parameters, It became Obvious that the sain on tie 
regression Farsmeters must he set very low in comrarison to 
the gain on the inderendent varisables»y in order to keer the 
regression rarameter estimates from varsinad faster than the 
estimates of the ainderendent variables. This iaimelies 
choosing small values for the moise variance of the 


res¢reassion rsrameters, Also» by khLeerins the gsin fairls low 
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om tne lesdinsg aumdicatory it was rossible to inmduce 3 Fhase 
lag tinst Brrroximatelys cancelled the lead. 

The requirement to keer gain low aim order to imrrove 
stability is evidently 3 conseauence of the increased desrees 
of freedom. The more rarameters to ne estimated» the more 
degrees of freedom in the model. Hish sain 1s analaesgous to 
relatively few date roints being used an resression. The 
more variables we introduce imto the model» the less sain we 
are able to use, 

Tt is indeed unfortunate that multirele regression 15 @ 
non-linear frrohlem when cast in 38 filter model, It would be 
useful to have 8&8 multirle regression model for which more 
recent observations were weishted more heavily than older 
ones im determining the regression rarameters. No doubt the 
innovative analyst could develor one to fit the srecific 
situstionr, Howevery clearly-defined techmiaues With 
damomstrated results are mot available to the rractitioner, 
The exrerts all have their favorite methodsy and much of the 
literature is difficult to read. There is clearly 3 meed for 


additional research in this erea. 


C. NON-FARAMETRIC FILTERING 


We close our discussion with em inmterestinsg alternative 
to conventional disital filtering techniaues. There sre 
those who 3re hothered Ay tne usual distributional 
assumetions made in any gerlication of rarametric statistics, 


Ar axtemns1ive litersture has develored im the field of 
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nom-Fsarametric statisticsy which 1s based on the fFrincirle 
thet distributional sssumetions are sasvoidedsy or at least 
Wweskened,. A strons Froint of mon-rParametric statistics 1646 
relative insensitivity to extreme outliers. Howevers little 
Frosress has been made im the monm-esrametric analysis of time 
series. AN excertion may be found in Tukey Cref.16]3» which 
is rresented in 3a highly intuitive manners with little or mo 
theoretical baecksround, 

One simrle ides advanced by Tukey is thet of median 
smoothing. The smoothed estimate is based on the median of 
several adJjscent data rointsy rather than om 3a weighted 
linear combination. The result is obviously 23 series of 
stersy since adiuacent dats roints will often have the same 
median. Tukey susdests several methods to restore some 
curvature im the estimate. These will mot be develored here. 
Tukey’s methods would be relatively hard to mechanize on a 
commuters hecsuse the methodoloss reauires extensive logical 
Mules. 

Tukey’s methods could he extended to real-time filtering 
rFroblems hw develoring 2 non-rarametric analogs to the 
recursive digital filter. Recall that the recursive digital 
filter consists of 3 weishted linear combination of recent 
data roints added to a weishted linear combination of recent 
estimates. The non-rarametric filter estimate would simels 
he the median of several recent dats Froints and several 
recemt states. The idesais intuitively sgrrealinsg and should 
he the subvect of future research. Discussion here will he 


limited to some of the more ohvious trars that await the 
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LIPIW STS 

ie such 3 filter were to he jesismeds the 
immeulse-resronse function would be meaningless» tbecause the 
median estimate would always be zero i1f the sran of the 
filter were greater than two, Hevertheless»s 3 median filter 
joes neve 2 freaquencs resronsery which im fact 165 82 
rarticularly masty one. 

Comsider 38 Sseven-Froint median smoothers where the state 
estimate at time t is the median of the measurements made sat 
mome <t-3) to time (tt3). This 1s analasous to the 
rectansular (rarametric) window discussed in Hammins Cref.7 J]. 
The rectangular window weishts a11 date Foints within the 
Window eaually. The median window ohfviously does the same, 
AS 3 results we would expect the freauency resronse of the 
median Window to have severe rirrles as does that of the 
rectangular window. We cam see intuitively that this is true. 
Since the sran of our examele median window 165 x7» the 
Pmeqtencs | Tresromse Of enw Treauency that is 8 non-zero 
imteger multirle of 1/77 1s obviously zero. The  freeauency 
resromse at zero freauency 165 ones since the zero freauericy 
imelies 8 constant. The amrlitude of the freauency resronse 
fells off to the first zero, then rises asain. Successive 
maxims decrease with incresasins freauency, but the freauency 
reSronse 16 always non-zero excert at freauencies that are 
nom-zero integer multirles of the recirprocsl of the sran of 
the window. Thussy the non-rarametric filter will meed to 
incorrorate some Cumsrecifiled) device to imerove the 


freaguemcy resrornse-. 
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Two other difficulties sre worthy of mention. Firsts» 
the sameling distribution of the median mays have 2a larser 
variance than the sameling distribution of the mean. This 
means that the Fersmetric filter mays Frovide 3 better 
estimate than the non-raerametric filter if the sssumetions an 
which the parametric filter is based are at all reasonable. 
Seconds, for the non-esarametric filter to he useful, the 
medisn must be 3a statistic of interest. If it 15 assumed 
that the distribution is symmetrics the median and mean ere» 
of courses equal. If the samelinsgd distribution is skewed, the 
mean cannot be deduced from the median unless strict 
Psrametric assumretions are imrosedy which of courses override 
the Justification for the non-rarametric filter in the first 
rlece. The ides is nevertheless iauntriguing, and should he 


exrelored further. 
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VI. SQME AFFLICATIONS 


The Kalman filter hes been @rrlied to Orerstions Research 
and economic Froblems with varying degrees of success, 
McWhorter Cref.17] conducted an emriricsel studs of the Kalman 
filter in which he comeared it to several other methods of 
time series forecasting. The results were mixed» with no 
method dominating. The Kalman filter comrared more favorably 
over a short term forecasting horizon than over 2 long term 
Ome. Its rerformance Wass moat surrrisinglys found to he 
degraded if the structural model was seriously mis-srecified. 
McWhorter rointed out some of the difficulties encountered in 
hullding the model. In an economic contexty it is often vers 
difficult to srecify the moise covariance matrices R and Qy 
and even to identify the structure of the state transition 
matrix q. The assumetions made are often sweerins and 
arbitrary, im comtrast to trackins arrlications where the 
moise Frocesses and esrecially the state transition model are 


relatively well understood. 


A. INVENTORY MANAGEMENT 


The Kslman filter is directly arrlicsable to imventors 
manssgements and if rFrorerly desismeds should be surerior to 
the finite exronential smoothing model of Bessler and Zenna 


Cref.14]. Downings Fikey and Morrison Cref.18] designed @2 





Kalman filter for the inventory control of rmuclear material. 
The rsrper is readebley and the filter is well-documented and 
ess3 to understand. Thes wse the concert of 2&8 control 
vectors which has not been mentioned here. An interesting 
reculiarity of the model is thet one of the messurements 15 
only available omece every twenty iterations. The stete 
transition matrix is 38 simele material balance relation which 
is obviously quite accurate. Such 2 model could be exrected 


to rerform auite well. 


B. ESTIMATING A MEAN FUNCTION 


Although the Kalman filter was derived from an assumption 
of stationsritsyy, we have seen that it can be aiuite rowerful 
in S@erarating 38 time varying sidgmal from moise. The examrles 
of section III were all essentislly estimates of the 
time~-varyvins mean function of 232 stochastic Frocess. The 

xamele rrocess was Gaussian with 28 constant variance. The 

Veriance Wes the messurement moisey and so directly 
influenced the Sain. If variance were not constants the 
Ferformance of 32 mnon-adaertive filter would he desreded., If 
the change im variance was great enoushy am adsartive filter 
would he reauired. 

A good method of estimating s time-varying mean funetion 
could he a@rrlied in rumerous areas» such as any sort of 
traffic or flow control froblems rerhars in auality control 
of large~-batch or flow manufacturing Frocessess and saris 


arrerlicstion where it is desirable to detect 3 chense in the 





FrOocess. The sensitivity of ives 1 Ce 16 directly 
adwustahnle hy the modeller throush the moise covariarice 
matrices Q snd Rk. 

A Frarticularly useful @rrlication would he to the 
estimation of the rate Faerameter of a non-time-homoseneous 
Foirssom -rocess. If this can be done accurately, the rrocess 
can be transformed to 32 stationary one Cref.é]9 which sreatly 
exrands the mumber of anmalstic tools that can be used. 

The Foissom Frocess 165 82 courntins Frocess im continuous 
timey and to attemet to filter a string of imterarrival time 
33ta would violate the samrlins theorem. The times of 
arrival are the measurement timesy and they are most 
certainly mot made at equally spaced intervals. Insteady the 
filter may be designed to samele a@ counting Frocess. At 
discrete aunterveals the filter would count the rumber of 
arrivals since some arbitrary time orisin. If the Frocess 
were to conmtinue for 32 lonst timey the time origin misht 
occasionally have to te reset to rrevent comruter overflow. 
It 18S esesy to see how this samrlinsg Frocess could be 
imelemented even if the inreut data were actually errival 
imstants im comtirnwous time. The samrling interval should be 
small enough that there is low Frobsbility that more then one 
arrival would occur during 23 given measurement interval. 
Sinee the mumber of arrivals 1s monotone mom-decressing if 
times 3 velocity or trend model would be a3arrrorrisate. The 
meut data would comsist of imntesgers. The state estimates 
Would mot. The monm-intesger estimate of mumber of srrivals ur 


Toecne Currert Cine would not be useful to us. Howevery the 
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second element of the state vectors tine velocity or trendy, 
would in fact be the filtered arrival rate estimate. Since 
the Frocess 15 moilsy and mon -Gaussiany a very Low 
steady-state gain 15 8BFrrorriate, 

The time-varsing Foissonm rrocess cannot have constant 
variances since the mean and the variance gre eaual. A low 
arrival rate imrelies high variance in the Foisson Frrocessys 
which 1s eauivalent to high measurement moisey which reauires 
low Sain. A constant-sgain filter would therefore he 
relatively more sensitive at low arrival rates than at high 
arrival rates. An adartive filter could he easily designed 
to use the inverse of the rate estimate as the measurement 
mo1se variance estimate. Stability might reauire that the 
adjustment of the measurement noise variance be itself 2 
filtering Frocess» im which the incoming variance estimate 1s 


regarded as date. 


Peet iPLeE REGRESSION 


If the redression constants are assumed known (Cor 
comeiuted by other means) the design of an @Frrrorriate filter 
1s Quite straightforward, and auslity of estimation is 
related directly to the auality of the model. Note thet the 
velocity filter is simely the regression of velocity or 
FOSition»s where the slore Farameter 1s known to he one. If 
the regression coefficients are assumed to vary in timey the 
Froblem becomes non-linear and is auite comrlex. RKecause of 


thie immense e@rrlicebility of this model» additionsl 
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develormental work 15s indeed 3 fertile field for future 


research. 


0D. SOME DESIGN CONSIDERATIONS 


In @Frlicsations where the moise covariance matrices R snd 
Qy and the system dynamic model (state transition matrix ) 
are known or easily estimated, design is straishtforward 
and has been successfully accomrlished while remaining in the 
time domain. However, ain arrlications where  sweerins 
Bassumetions sare reauireds 2 freauency-domain analysis could 


be very helrful. Some guidelines sre as follows: 


1. Srectral Analysis of the Dats 
A srectral analysis of samrle dats will show what the 


frequenmcs resronse of the filter should be. The Fast Fourier 


Transform (FFT) frosram availsble im most commuter libraries 


4 


1s sgeneraglly easy to use. Howevery the FFT Frosrams 
ganerally reauire an exact rower of 2 for the number of data 
FOints. Hamming Cref.7] roints out some ritfalls. Since 
stationarity is assumed, the data should be considered as 2 
rotating cylinders and if the starting end endings values are 
mot similary 2 discontinuity will exist in the srectrum. The 
data can he tarered and padded with zerosys but exactly the 
best method to accomrlish this is unknown. Several methods 
might be tried, 

The main virtue of the FFT is its speed. It works well 


on |e lons run of dstea. If the number of dats roints is small 
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(around 3a hundred) 1t misht be effective to find (Cor write) 2 
less efficients conventional discrete Fourier transform 
Frogramy which would mot reauire Fraeddedy truncateds or 
tarered data if the starting and ending values ere similar, 
If the FFT frosram used does not reauire an exact rower of 
two for the number of ineut data Frointsy it would be well to 
find out why not. The frrogrem may be doing the radding and 
tsrering itself» and the analyst should be curious a5 to how. 
The analyst should remember that the srectrum 15 commuted 
from the detey and it is therefore an estimate. If the run 
of data is short, there will he considerahle variance in the 


estimate. 


ee Freauencys Analysis of Frorosed Models 

The analyst may test the effect of assumrtions made 
in desisgning the filter oy simely obtaining an aimrFulse 
resronse of the filter and running at throusn an FFT. 
Truncation and tarering is no Froblems because the imrulse 
resronse will a3arrroach zero with time. The Frorer imreulse 
funetion 15 Ssimely 3a 1 followed by at zeros for a filtery 
or ai in the middle of 2" -1 zeros for a smoother. If the 
OutePut of the FFT consists of real and imasinary comronentsy 


1t will be necessary to comrute smelitude and rhase. 


S Adjusting the Model 


If tne model dynamics seem adeauste bout the bandwidth 
1S wromsy the analyst should bys mow have some iunsisht into 


whet adidjustments to make to the noise covaeriance matrices to 





try to imrrove things. In 38 model of anv comrlexity at sll» 
there 3re numerous rossible combinations. Howevery, even some 
Imerovement over the imitisl assumetions will be beneficial. 
We sre not lookins for theoretical elesances we sre looking 
for rerformance. 

Ferhars the model dunamics obviously call for 2 trend 
filter or even a chanse-of-trend Cacceleration) filters but 
the data 15 quite moisy, Consideration should be siven to 
lowering the order of the filter. A very Low-Ssainm velocity 
filter will mot follow changes ain trend well. A hisher-sain 
scalar filter mays do so more effectively, although it will 
lsse ses steady trend. There sre mans tradeoffs» and we cannot 


schieve rerfection. 


a, Testing the Model 
The model should be tested on resl or simulated dats. 


From here omy the modelling Frocess is the standard cyclical 
oney Go1ms peck. to earlier sters 35 mecessary wnitil 


satisfactory rerformance 1S scnieved,. 
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AFFENDUIX A. DUERIVATIONS 
1. SCALAR KALMAN FILTER 


8+ Recursive Formula for Kalan Gain 
The covariance extraerolation eauation 
F(t) = §@ E(t-1) GF + Q 
reduces im the scalar case to 
P(t) = Z2(t-1) + Q 
Since 
K(t) = £(t) HR” 
we mas Writes for the scalar casey 
&(t-1) = K(t-1) R 
Similarly» since 
K(t) = PHC MPH + RI” 
by reducing to the scalar case and substituting» we may write 
K¢t) = Metco in. tf @ = NGte 1) ote Wek 
INnct=— 1) Uk =+ QW + °K Meee i tun + ] 
Ob. Stesdu-State Kalman Gain 
Rearransging the recursive gain eauation and letting 
mC iC ta=1) = 
we see that 
K+ (Q/RIK - Q/R = 0 


Ky the auadratic formuleas 





We sre obliged to take the larser rooty since the smaller 
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root would force the s3sim to be negative. We also observe the 


imverse relationship 


za 
Cy en ee 4k 7+ OR 
ROC Rt et eOa 7ans = 0°74R + O/R 
me E 
ay Rik 


Vee C1 


ec. Transient Kalman Gain 
Recall that the Kalman filter reauires 38 frior state 
estimate X(0) and 3@ Frior estimate of covariance FCO). This 
reatuuirement can be avoided by using KC1) = Ly which sllows 
the aimitial state estimate to he eeaual to the first 
measurement. Recall that 
Cee i Re 


Since K(1) = tl» then 2¢1) = Rk 


ae Ammzlitude and Fhase of Freauency Resronse 


The freauencys resronse is 
ow 
; t 
Hiv) = 2 Co exe (-1V) I 
tzo 


Since 
lin expe (-iv)| = OL 
tnen 
H¢v) = 38 / EF1-b exrt-iv)d 
The amrPlitude sausared is 


2 
Ges a7 Eieh exe (-iv)] Ci-b exp Civ)] 
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Gy Euler’s relation 
A = at/ C1 + & -2o cos vi 
Which may ne written 
A = a / C(i-b) + 2bC1-cos v)J 
recelling that 
Q/R = K*/(C1-K) = a7/b 


We mas write the amelitude as 


A = Q/ KK 
Oi eel -Ccos ¥v) 
The rhase ansle 1s 
Ov) = srpetem Elm(vy) 7 Ketv) J 
where Im(v) and Re€v) e2Bre the imasinary and real rarts of 
Hivdy which may be written 


Hiv) = 3 L-i-h e3 
Li—-b exe C-iv)JCi-b ex: 


1 


Hev) = 3 (1tb cos v ~-1b sim v) 
1 + O “~-b cos v 
Which allows ws to write 


Ge Sretam £6 sift Vi 7 C1l-b fos v) I 


The ansle for maximum -rhese shift occurs when 


2 
HOLY es Desh eos Y¥ ae @) 
EE ———— 

dy 1+ b7 -2h cos v 


so that the maximum rhase shift O(vimax occurs when 
Vos grcecos 


and nas 3 value of 


Stvimax = aretan —-b sintarcenas oO) 
l-b cosCarccos b) 


= apetan (€ -b / /1~-t*) 
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eee ULSE RESPONSE FUNCTIONS 


The imreulse-resronse of the scalar Kalman filter 1s 


g(t) = ae tes Os le2s..+ 


Se Imrulse-resronse of Nouble Filter 


s(t) = ab @® ab 


Es t 
b ~ -h 
Ge = 3b Ben ek ne 
Az-0 bio 


Mee eet a* be 


Db. Imreulse-resronse of Scalar Kalman Smoother 


=e 
Ce oa GX) ab 


oo 7 ao 
b tech 2am 
a(t) = Sadat « @Zo = Xo? Do 
s°*0O bB=0 bro 


2 
Gey, len eee 7 (1 tb) 
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